In this paper, we propose to pre-train audio encoders using synthetic patterns instead of real audio data. Our proposed framework consists of two key elements. The first one is Masked Autoencoder (MAE), a self-supervised learning framework that learns from reconstructing data from randomly masked counterparts. MAEs tend to focus on low-level information such as visual patterns and regularities within data. Therefore, it is unimportant what is portrayed in the input, whether it be images, audio mel-spectrograms, or even synthetic patterns. This leads to the second key element, which is synthetic data. Synthetic data, unlike real audio, is free from privacy and licensing infringement issues. By combining MAEs and synthetic patterns, our framework enables the model to learn generalized feature representations without real data, while addressing the issues related to real audio. To evaluate the efficacy of our framework, we conduct extensive experiments across a total of 13 audio tasks and 17 synthetic datasets. The experiments provide insights into which types of synthetic patterns are effective for audio. Our results demonstrate that our framework achieves performance comparable to models pre-trained on AudioSet-2M and partially outperforms image-based pre-training methods.

Running short on images to train your model? Here's how to increase your dataset size multi-fold with synthetic images using Image… It is quite a known fact that the quality and quantity of dataset used to train your model will have a direct effect on the model's performance. Availability of an extensive, versatile dataset could seal the deal, and you could jump to the next step of your Machine Learning / Deep Learning pipeline. But often you might end up in a situation where the dataset you need is just not readily available. Or the data you have is not enough to train a heavy duty model.

Deep neural networks are known to achieve superior results i n classification tasks. However, it has been recently shown that they are incapable t o detect examples that are generated by a distribution which is different than the one they have been trained on since they are making overconfident prediction fo r Out-Of-Distribution (OOD) examples. OOD detection has attracted a lot of attenti on recently. In this paper, we review some of the most seminal recent algorit hms in the OOD detection field, we divide those methods into training and po st-training and we experimentally show how the combination of the former with t he latter can achieve state-of-the-art results in the OOD detection task. Since the seminal work of Krizhevsky et al. (2012), Deep Neur al Networks (DNNs) have demonstrated great success in several applications, e.g.